Multidimensional Range Queries on Modern Hardware

Range queries over multidimensional data are an important part of database workloads in many applications. Their execution may be accelerated by using multidimensional index structures (MDIS), such as kd-trees or R-trees. As for most index structures, the usefulness of this approach depends on the selectivity of the queries, and common wisdom told that a simple scan beats MDIS for queries accessing more than 15%-20% of a dataset. However, this wisdom is largely based on evaluations that are almost two decades old, performed on data being held on disks, applying IO-optimized data structures, and using single-core systems. The question is whether this rule of thumb still holds when multidimensional range queries (MDRQ) are performed on modern architectures with large main memories holding all data, multi-core CPUs and data-parallel instruction sets. In this paper, we study the question whether and how much modern hardware influences the performance ratio between index structures and scans for MDRQ. To this end, we conservatively adapted three popular MDIS, namely the R*-tree, the kd-tree, and the VA-file, to exploit features of modern servers and compared their performance to different flavors of parallel scans using multiple (synthetic and real-world) analytical workloads over multiple (synthetic and real-world) datasets of varying size, dimensionality, and skew. We find that all approaches benefit considerably from using main memory and parallelization, yet to varying degrees. Our evaluation indicates that, on current machines, scanning should be favored over parallel versions of classical MDIS even for very selective queries

Efficient Processing of Range Queries in Main Memory

Datenbanksysteme verwenden Indexstrukturen, um Suchanfragen zu beschleunigen. Im Laufe der letzten Jahre haben Forscher verschiedene Ansätze zur Indexierung von Datenbanktabellen im Hauptspeicher entworfen. Hauptspeicherindexstrukturen versuchen möglichst häufig Daten zu verwenden, die bereits im Zwischenspeicher der CPU vorrätig sind, anstatt, wie bei traditionellen Datenbanksystemen, die Zugriffe auf den externen Speicher zu optimieren. Die meisten vorgeschlagenen Indexstrukturen für den Hauptspeicher beschränken sich jedoch auf Punktabfragen und vernachlässigen die ebenso wichtigen Bereichsabfragen, die in zahlreichen Anwendungen, wie in der Analyse von Genomdaten, Sensornetzwerken, oder analytischen Datenbanksystemen, zum Einsatz kommen. Diese Dissertation verfolgt als Hauptziel die Fähigkeiten von modernen Hauptspeicherdatenbanksystemen im Ausführen von Bereichsabfragen zu verbessern. Dazu schlagen wir zunächst die Cache-Sensitive Skip List, eine neue aktualisierbare Hauptspeicherindexstruktur, vor, die für die Zwischenspeicher moderner Prozessoren optimiert ist und das Ausführen von Bereichsabfragen auf einzelnen Datenbankspalten ermöglicht. Im zweiten Abschnitt analysieren wir die Performanz von multidimensionalen Bereichsabfragen auf modernen Serverarchitekturen, bei denen Daten im Hauptspeicher hinterlegt sind und Prozessoren über SIMD-Instruktionen und Multithreading verfügen. Um die Relevanz unserer Experimente für praktische Anwendungen zu erhöhen, schlagen wir zudem einen realistischen Benchmark für multidimensionale Bereichsabfragen vor, der auf echten Genomdaten ausgeführt wird. Im letzten Abschnitt der Dissertation präsentieren wir den BB-Tree als neue, hochperformante und speichereffziente Hauptspeicherindexstruktur. Der BB-Tree ermöglicht das Ausführen von multidimensionalen Bereichs- und Punktabfragen und verfügt über einen parallelen Suchoperator, der mehrere Threads verwenden kann, um die Performanz von Suchanfragen zu erhöhen.Database systems employ index structures as means to accelerate search queries. Over the last years, the research community has proposed many different in-memory approaches that optimize cache misses instead of disk I/O, as opposed to disk-based systems, and make use of the grown parallel capabilities of modern CPUs. However, these techniques mainly focus on single-key lookups, but neglect equally important range queries. Range queries are an ubiquitous operator in data management commonly used in numerous domains, such as genomic analysis, sensor networks, or online analytical processing. The main goal of this dissertation is thus to improve the capabilities of main-memory database systems with regard to executing range queries. To this end, we first propose a cache-optimized, updateable main-memory index structure, the cache-sensitive skip list, which targets the execution of range queries on single database columns. Second, we study the performance of multidimensional range queries on modern hardware, where data are stored in main memory and processors support SIMD instructions and multi-threading. We re-evaluate a previous rule of thumb suggesting that, on disk-based systems, scans outperform index structures for selectivities of approximately 15-20% or more. To increase the practical relevance of our analysis, we also contribute a novel benchmark consisting of several realistic multidimensional range queries applied to real- world genomic data. Third, based on the outcomes of our experimental analysis, we devise a novel, fast and space-effcient, main-memory based index structure, the BB- Tree, which supports multidimensional range and point queries and provides a parallel search operator that leverages the multi-threading capabilities of modern CPUs

qwLSH: Cache-conscious Indexing for Processing Similarity Search Query Workloads in High-Dimensional Spaces

Similarity search queries in high-dimensional spaces are an important type of queries in many domains such as image processing, machine learning, etc. Since exact similarity search indexing techniques suffer from the well-known curse of dimensionality in high-dimensional spaces, approximate search techniques are often utilized instead. Locality Sensitive Hashing (LSH) has been shown to be an effective approximate search method for solving similarity search queries in high-dimensional spaces. Often times, queries in real-world settings arrive as part of a query workload. LSH and its variants are particularly designed to solve single queries effectively. They suffer from one major drawback while executing query workloads: they do not take into consideration important data characteristics for effective cache utilization while designing the index structures. In this paper, we present qwLSH, an index structure for efficiently processing similarity search query workloads in high-dimensional spaces. We intelligently divide a given cache during processing of a query workload by using novel cost models. Experimental results show that, given a query workload, qwLSH is able to perform faster than existing techniques due to its unique cost models and strategies.Comment: Extended version of the published wor

Smart framework for real-time monitoring and control of subsurface processes in managed aquifer recharge applications: project outlook

International audienc

A quantitative analysis of stratospheric HCl, HNO3, and O3 in the tropopause region near the subtropical jet

The effects of chemical two-way mixing on the Extratropical Transition Layer (ExTL) near the subtropical jet are investigated by stratospheric tracer-tracer correlations. To this end, in situ measurements were performed west of Africa (25–32◦N) during the Transport and Composition of the Upper Troposphere Lower Stratosphere (UTLS)/Earth System Model Validation (TACTS/ESMVal) mission in August/September 2012. The Atmospheric chemical Ionization Mass Spectrometer sampling HCl and HNO3 was for the first time deployed on the new German High Altitude and Long range research aircraft (HALO). Measurements of O3, CO, European Centre for Medium-Range Weather Forecasts (ECMWF) analysis, and the tight correlation of the unambiguous tracer HCl to O3 and HNO3 in the lower stratosphere were used to quantify the stratospheric content of these species in the ExTL. With increasing distance from the tropopause, the stratospheric content increased from 10% to 100% with differing profiles for HNO3 and O3. Tropospheric fractions of 20% HNO3 and 40% O3 were detected up to a distance of 30 K above the tropopause

Genetic Characterization of Listeria from Food of Non-Animal Origin Products and from Producing and Processing Companies in Bavaria, Germany

Reported cases of listeriosis from food of non-animal origin (FNAO) are increasing. In order to assess the risk of exposure to Listeria monocytogenes from FNAO, the genetic characterization of the pathogen in FNAO products and in primary production and processing plants needs to be investigated. For this, 123 samples of fresh and frozen soft fruit and 407 samples of 39 plants in Bavaria, Germany that produce and process FNAO were investigated for Listeria contamination. As a result, 64 Listeria spp. isolates were detected using ISO 11290-1:2017. Environmental swabs and water and food samples were investigated. L. seeligeri (36/64, 56.25%) was the most frequently identified species, followed by L. monocytogenes (8/64, 12.50%), L. innocua (8/64, 12.50%), L. ivanovii (6/64, 9.38%), L. newyorkensis (5/64, 7.81%), and L. grayi (1/64, 1.56%). Those isolates were subsequently sequenced by whole-genome sequencing and subjected to pangenome analysis to retrieve data on the genotype, serotype, antimicrobial resistance (AMR), and virulence markers. Eight out of sixty-four Listeria spp. isolates were identified as L. monocytogenes. The serogroup analysis detected that 62.5% of the L. monocytogenes isolates belonged to serogroup IIa (1/2a and 3a) and 37.5% to serogroup IVb (4b, 4d, and 4e). Furthermore, the MLST (multilocus sequence typing) analysis of the eight detected L. monocytogenes isolates identified seven different sequence types (STs) and clonal complexes (CCs), i.e., ST1/CC1, ST2/CC2, ST6/CC6, ST7/CC7, ST21/CC21, ST504/CC475, and ST1413/CC739. The core genome MLST analysis also showed high allelic differences and suggests plant-specific isolates. Regarding the AMR, we detected phenotypic resistance against benzylpenicillin, fosfomycin, and moxifloxacin in all eight L. monocytogenes isolates. Moreover, virulence factors, such as prfA, hly, plcA, plcB, hpt, actA, inlA, inlB, and mpl, were identified in pathogenic and nonpathogenic Listeria species. The significance of L. monocytogenes in FNAO is growing and should receive increasing levels of attention

Lagrangian matches between observations from aircraft, lidar and radar in a warm conveyor belt crossing orography

Warm conveyor belts (WCBs) are important airstreams in extratropical cyclones, often leading to the formation of intense precipitation and the amplification of upper-level ridges. This study presents a case study that involves aircraft, lidar and radar observations in a WCB ascending from western Europe towards the Baltic Sea during the Hydrological Cycle in the Mediterranean Experiment (HyMeX) and T-NAWDEX-Falcon in October 2012, a preparatory campaign for the THORPEX North Atlantic Waveguide and Downstream Impact Experiment (T-NAWDEX). Trajectories were used to link different observations along the WCB, that is, to establish so-called Lagrangian matches between observations. To this aim, an ensemble of wind fields from the global analyses produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble of Data Assimilations (EDA) system were used, which allowed for a probabilistic quantification of the WCB occurrence and the Lagrangian matches. Despite severe air traffic limitations for performing research flights over Europe, the German Aerospace Center (DLR) Falcon successfully sampled WCB air masses during different phases of the WCB ascent. The WCB trajectories revealed measurements in two distinct WCB branches: one branch ascended from the eastern North Atlantic over southwestern France, while the other had its inflow in the western Mediterranean. Both branches passed across the Alps, and for both branches Lagrangian matches coincidentally occurred between lidar water vapour measurements in the inflow of the WCB south of the Alps, radar measurements during the ascent at the Alps and in situ aircraft measurements by Falcon in the WCB outflow north of the Alps. An airborne release experiment with an inert tracer could confirm the long pathway of the WCB from the inflow in the Mediterranean boundary layer to the outflow in the upper troposphere near the Baltic Sea several hours later. The comparison of observations and ensemble analyses reveals a moist bias in the analyses in parts of the WCB inflow but a good agreement of cloud water species in the WCB during ascent. In between these two observations, a precipitation radar measured strongly precipitating WCB air located directly above the melting layer while ascending at the southern slopes of the Alps. The trajectories illustrate the complexity of a continental and orographically influenced WCB, which leads to (i) WCB moisture sources from both the Atlantic and Mediterranean, (ii) different pathways of WCB ascent affected by orography, and (iii) locally steep WCB ascent with high radar reflectivity values that might result in enhanced precipitation where the WCB flows over the Alps. The linkage of observational data by ensemble-based WCB trajectory calculations, the confirmation of the WCB transport by an inert tracer and the model evaluation using the multi-platform observations are the central elements of this study and reveal important aspects of orographically modified WCBs.</p

Relativistic, QED and nuclear effects in highly charged ions revealed by resonant electron-ion recombination in storage rings

Dielectronic recombination (DR) of few-electron ions has evolved into a sensitive spectroscopic tool for highly charged ions. This is due to technological advances in electron-beam preparation and ion-beam cooling techniques at heavy-ion storage rings. Recent experiments prove unambiguously that DR collision spectroscopy has become sensitive to 2nd order QED and to nuclear effects. This review discusses the most recent developments in high-resolution spectroscopy of low-energy DR resonances, experimental studies of KLL DR of very heavy hydrogenlike ions, isotope shift measurements of DR resonances, and the experimental determination of hyperfine induced decay rates in divalent ions utilizing DR.Comment: 6 pages, 5 figures, proceedings of the 4th Conference on Elementary Processes in Atomic Systems, Cluj-Napoca, Romania, June 18-20, 2008, accepted for publication in Nucl. Instrum. Methods